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Abstract: In an insightful commentary, the author ruminates on the dilemmas of 
coaching in the context of high-stakes testing. 



Essay: 

This is surely the age of accountability, both in higher education and in K-12. As the 
stakes associated with test performance rise, both for the students and for their 
institutions, the dilemmas of coaching for test performance have risen as well. Many of 
the newer forms of assessment make the distinctions between teaching and assessment 
less and less clear. 

When the test takes the form of a dissertation, a portfolio or an architectural design, can 
we tell how much was done by the coach and how much by the candidate? What if one's 
peers have collaborated in producing the artifact, whether as co-designers or as editors 
or critics? And if we insist on tests that reduce dramatically the role of mentors or 
coaches, might we also be using tests that are more highly artificial and 
unrepresentative? 

Here are some ruminations about the dilemmas of coaching in the contexts of high-stakes 
testing. They are drawn from an essay titled "Send me in, Coach!" that I've written for a 
future issue o/The New Teacher, a journal published by the City University of New York. 

Lee S. Shulman 
May 2008 



When Coaching and Testing Collide 



It's a scene we have watched dozens of times in the movies. A young man or woman of 
modest talent tries out for the baseball or football or basketball team under the tutelage of 
a gruff, demanding coach who expresses initial doubts about the likelihood that the kid 
will prove himself or herself worthy of a spot on the team. The coach is tough and 
persistent, setting high standards and then mercilessly driving all his charges to meet 
them. In the climactic scene at the season’s end, the good guys or gals are losing by 
several baskets, or runs, or a touchdown — depending on the sport. "Send me in, coach," 
pleads our young hero/ine, which coach reluctantly does. The kid scores the winning 
points, and the team wins. The coach turns out to have a heart of gold, and the reasons for 
his seeming cruelty become apparent. 

What exactly is it that the coach provides the aspirant? Let me propose five processes 
associated with both the coach and mentor roles: 1) technique, learned through endless 
drill; 2) strategy, that allows the person who is coached to become capable of a 
conception of the work that will turn out to be pivotal in their eventual victory; 3) 
motivation, which produces a "Rocky-like" level of commitment that will help them 
exceed their own and others' expectations; 4) vision, where players come together in a 
new vision of the process and their capabilities for success; and 5) identity, whereby the 
protagonist not only wins, but is transfonned, with an internalized new sense of self. 

In sports there is always a clear line between the coaching situation and the performance 
context. When the final jump shot is made from the three-point line by the basketball 
player, the coach can’t jump onto the court and give the ball the extra momentum or spin 
it might need. I prefer to call such typical relationships between a coach/mentor and 
player/protege examples of unmediated mentoring. No separate product comes in the 
middle between the coaching and the performing that renders the relative contributions of 
the coach and the coached inherently ambiguous because the entire perfonnance is visible 
and is itself the basis for evaluating success or failure. 

There is, however, an entire genre of mediated mentoring. The performance is not 
directly observed and has yielded a product which is the focal point of competition and 
evaluation. Thus in the case of mediated performances, the respective roles of coach and 
perfonner are inherently invisible. Although the five processes are in place and just as 
transformative, there is inherently no way to discern how much of the work was done 
independently by the candidate, by peers or by advisors. 

Whenever mentoring is mediated by a product whose actual authoring processes are not 
directly observable, as is the case with literature, objects of architectural or mechanical 
design, scholarly publications, doctoral dissertations, and even paintings, assessment of 
individual competence is problematic. But are these problems of educational 
measurement or a new set of realities regarding the conditions of expert performance? 
Stanford education professor Sam Wineburg and others point out that the crux of the 
problem may not be measurement error but rather the inherently social and interactive 
character of the performances whose competence is assessed. Writing is and should be 




critiqued and edited, as should painting, the designs for buildings and the research 
performed in scientific laboratories. To avoid mentoring merely to ensure the legitimacy 
of individual test scores might even be judged a form of malpractice! So we are faced 
with an essential tension between the inherently social character of most forms of 
complex human performance and the psychometric imperative to estimate a "true score" 
for ability or any other personal trait using the individual as the unit of analysis. 

In an education setting, the distinction between the scores that a student earns on any test- 
like event — multiple choice test, essay exam, portfolio or senior sermon in a seminary — 
and their underlying "true" capability is a reflection of the distinction, borrowed perhaps 
from the field of linguistics, between competence and performance. Psychometrics rests 
on the claim that the observed performance is a valid indicator if it tracks the underlying 
competence faithfully. But what if mentored or coached performances actually track 
underlying competence more validly than measurement of students working alone? What 
if the composition written by a student in the presence of his editing team is a better 
indicator of his future writing competence than having him write alone? 

That is what sits at the heart of the puzzle. 

My proposal for "getting over" this essential tension is three-fold: making changes in the 
processes of assessment, making explicit the parameters of mentoring, and developing a 
clear code of ethical principles for both assessment and mentoring. At the heart of these 
proposals is the principle of transparency. Everything possible must be done to ensure 
that the roles of mentors, peers and students be transparently clear in any mediated 
mentoring activity. There should be ways of reporting on the character of coaching for 
test performance that make the efforts of the coach entirely transparent to assessment. 

I have often written that collaboration is a marriage of insufficiencies', that students can 
work together in ways that scaffold and support each others' learning, and in ways that 
support each others' knowledge. Now I call for a marriage of sufficiencies to overcome 
the essential tensions between individual work and collaborative performance, coaching 
support and independent assessment, the mentor as an agent of zealous advocacy and the 
mentor as a steward of the commons. 



As Dewey observed, we will not solve this problem, we will get over it. It is built in to 
the psychometric paradox: Our measurement models are psychometric but our assessment 
needs are often sociomctxic, requiring the measurement of socially scaffolded and joint 
productions. 
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